Noninvasive imaging-based machine learning algorithm to identify progressive disease in advanced hepatocellular carcinoma receiving second-line systemic therapy

The aim of this study was to predict tyrosine kinase inhibitors (TKI) plus anti-PD-1 antibodies (TKI-PD-1) efficacy as second-line treatment in advanced hepatocellular carcinoma (HCC) using radiomics analysis. From November 2018 to November 2019, a total of 55 patients were included. Radiomic features were obtained from the CT images before treatment and filtered using intraclass correlation coefficients (ICCs) and least absolute shrinkage and selection operator (LASSO) methods. Subsequently, ten prediction algorithms were developed and validated based on radiomic characteristics. The accuracy of the constructed model was measured through area under the receiver operating characteristic curve (AUC) analysis; survival analysis was performed via Kaplan–Meier and Cox regression analyses. Overall, 18 (32.7%) out of 55 patients had progressive disease. Through ICCs and LASSO, ten radiomic features were entered into the algorithm construction and validation. Ten machine learning algorithms showed different accuracies, with the support vector machine (SVM) model having the highest AUC value of 0.933 in the training cohort and 0.792 in the testing cohort. The radiomic features were associated with overall survival. In conclsion, the SVM algorithm is a useful method to predict TKI-PD-1 efficacy in patients with advanced HCC using images taken prior to treatment.

www.nature.com/scientificreports/ study involving patients with unresectable HCC (uHCC) who had failed sorafenib treatment 10 . They discovered that a combination of TKI and PD-1 inhibitors (TKI-PD-1) was more advantageous than TKI monotherapy for these patients, indicating that TKI-PD-1 therapy could be a promising treatment option for advanced HCC after first-line failure. However, a subset of patients still progress during TKI-PD-1 therapy, and thus the identification of biomarkers to predict each patient's response to combination therapy is essential. To date, serum and tissue sample analyses have not clearly defined the subpopulation most likely to progress 2 . In addition to collecting invasive serum and tissue samples, noninvasive imaging is routinely performed during routine medical examinations. Radiomics comprise valuable information sources for prognostication 11 . Machine learning can automatically construct models to interpret medical images based on their radiomic features dataset. A predictive approach using CT-based radiomics is a noninvasive, cost-effective way to identify patients at high risk of PD during TKI-PD-1 therapy. However, such machine learning methods need further study before applying radiomics in the clinic. Therefore, this study was carried out to construct and compare machine learning models based on radiomics for predicting the response of advanced HCC to combined TKI-PD-1 treatment as second-line treatment.

Results
Patient population. A total of 55 patients were enrolled in the study, including 37 (67.3%) patients with non-progression disease (non-PD) and 18 (32.7%) patients with PD. The PD patients experienced a shorter overall survival (OS) than the non-PD patients (9.35 months versus not reached, P < 0.001, Fig. 1a). The median patient age in the overall setting was 53.0 years (range, 43.0-55.0), and most patients were male (90.9%). The etiology of hepatitis B HCC accounted for 94.5%, and 67.3% of patients had an alpha fetoprotein level > 200 ng/ mL. Their mean albumin-bilirubin (ALBI) score was − 2.60, which was relatively higher in PD patients than in non-PD patients (− 2.29 versus − 2.76, P < 0.001). Median platelet (PLT), aspartate transaminase (AST), alanine transaminase (ALT), and alkaline phosphatase (ALP) levels were 194.00*10 9 /L, 60.30 U/L, 44.50 U/L, and 134.10 U/L, respectively. At baseline, 61.8% and 38.2% of patients had multiple tumors and a single nodule, respectively. The tumor diameter was > 5 cm in 81.8% of patients and < 5 cm in 18.2%. The proportions of patients with macroscopic vascular invasion and extrahepatic spread at baseline were 80% and 34.5%, respectively. Characteristic differences were not reported for these variables (P > 0.05) except for ALBI (Table 1).
Radiomics feature selection and analysis. In total, 2,458 features were acquired for every patient, among which 568 features were excluded with intraclass correlation coefficients (ICCs) < 0.90, leaving 1,890 features for further analysis. Then, 10 discriminatory wavelet-related features between the non-PD and PD patients  Table S1). After obtaining these 10 features, we further explored their relationships using Spearman correlation analysis. The Chord diagram shows links between the features (shown in Fig. 1c). Obviously, warm red wide bands indicated feature 1 and feature 2, feature 3 and feature 5, and feature 5 and feature 6 were significantly positively related to each other in the network. Meanwhile, features 4 and 6 were negatively related to the green wide band connection. Overall, the detailed relationships among features are shown in heatmap Fig. 1c, and feature 1 and feature 2 had the highest correlation coefficient (0.99) in Spearman analysis. We next performed a principal component analysis (PCA) process to compress the features and reduce the dimensionality. As visualized in Fig. 1d, the PD patients (left bottom cluster) could be separated from the non-PD patients (right top cluster).
Differential values were distributed between PD and non-PD patients for all radiomics features (Supplementary Figure S1). Higher expression was noted for features 1, 2, 4, 7, and 10 in non-PD patients, and the difference for feature 1 was significant (P < 0.05). In contrast, PD patients showed relatively high expression of features 3, 5, 6, 8, and 9, but the difference was not significant (P > 0.05). The area under the receiver operating characteristic curve (AUC) for the prediction of PD ranged from 0.572 to 0.664 for the ten features, as shown in Supplementary Figure S2. Among these features, the AUC of feature 4 was 0.661 (95% CI: 0.500-0.822) and the AUC of 6 was 0.664 (95% CI: 0.509-0.818), both of which reached significance in ROC analysis (P < 0.05).
The AUC values of each model were generally higher in the training cohort than in the test cohort (Fig. 3). Different accuracies of the models were exhibited, among which the SVM model had the highest AUC score of 0.933 in the training cohort and 0.792 in the testing cohort. The SVM, Bayes, and bagging algorithms all achieved AUCs > 0.7 in the test cohort.
Survival analysis. Overall, 19 patients died during the follow-up. OS was not reached in all patients, and the OS in subgroups based on the drugs used is presented in Supplementary Figure S3, which shows no significant difference among subgroups according to the drugs used (P = 0.990). The log-rank test was used to compare high versus low radiomic features with survival. Four out of ten features were found to be significantly associated with OS, including feature 4 (P = 0.039), feature 5 (P = 0.049), feature 8 (P = 0.018), and feature 9 (P = 0.041), as shown in Supplementary Figure S4. Cox regression analysis incorporating radiomic features and clinical variables was conducted to identify survival-associated factors (Fig. 4a). A total of 7 factors (features 4, 6, 8, ALBI, tumor diameter, ALT, and portal vein invasion) generated in the univariate Cox regression model were further selected to be included in the multivariate Cox regression model. Finally, feature 4 (P = 0.002) and feature 6 (P = 0.001) were favorable survival factors, while feature 8 (P = 0.033), ALBI (P = 0.032), and portal vein invasion (P = 0.002) were hazardous survival factors in the multivariate Cox regression model with a c-index of 0.81 (Fig. 4b). Figure 4c shows the more net benefit of using a model comprising clinical characteristics plus radiomics than using a clinical model.

Discussion
The present study demonstrated that radiomic features extracted from initial CT images before TKI-PD-1 treatment could predict the HCC treatment response. The radiomics-based SVM machine learning algorithm using ten LASSO filtered features predicted the PD response with a mean accuracy, sensitivity, specificity, precision, and F1 score of 81.8%, 100.0%, 43.7%, 78.8%, and 88.0%, respectively, in the training cohort and 69.1%, 95.0%, 20.0%, 70.6%, and 80.0%, respectively, in the testing cohort. Treatment options for advanced HCC have rapidly evolved over the past several years. The optimal course of treatment for advanced HCC patients who do not respond to first-line therapy is currently not well established 12,13 . Since TKIs target vascular endothelial growth factor receptors (VEGFRs) and modulate the immune microenvironment, the treatment efficacy of PD-1 inhibitors would be theoretically improved when combined with TKIs 14 . Indeed, emerging clinical data suggest that although a single agent has demonstrated disease control with manageable toxicity, improved synergistic outcomes have been shown. Lenvatinib combined with pembrolizumab resulted in a DCR of > 85% in a recent phase Ib study 2 . In a phase II trial RESCUE, the DCR was > 77% in the first-line and 75% in the second-line advanced HCC cohort treated with camrelizumab combined with apatinib 15 . Additionally, a recent study demonstrated that a combination of TKI-PD-1 was found to be more advantageous than TKI monotherapy for HCC patients who had previously failed sorafenib treatment 10 . This suggests that TKI-PD-1 therapy could be a promising treatment option for advanced HCC after first-line therapy failure.
The additional refinement of the subpopulation of patients most likely to benefit from this combination therapy would be of clinical significance. However, reliable prediction tools to support the precise therapy of TKI-PD-1 are not currently available and are urgently needed in the era of immunotherapy. To date, predictive markers of response or resistance using serum or tissue samples as biomarkers for advanced HCC have not been clearly defined, including PD-L1 expression and tumor mutation burden (TMB) levels 16,17 . Recently, Yang et al. found that copy number variations (CNVs) in plasma cell-free DNA (cfDNA) could predict the clinical result of combined PD-1 inhibitor and lenvatinib therapy and other immune checkpoint inhibitor-based therapies in hepatobiliary cancers 18 . However, further study is warranted to verify the clinical value of plasma cfDNA in HCC. Thus, further research to identify new possible methods to predict the treatment response to TKI-PD-1 is necessary.
Radiomics involves applications of computer vision and artificial intelligence to investigate the hidden characteristics of radiographic images in a quantitative manner 19 . CT images of tumors contain a large amount of useful information that generally cannot be recognized by physicians simply looking at them 20 . The expression of immunotherapy target PD-L1 at the protein level, as well as PD-1 and CTLA4 at the mRNA level, was found by Hectors et al. to be correlated with radiomic features of HCC 21 . Radiomics also showed a powerful ability to predict CD8 + T-cell infiltration of HCC in the study of Liao et al. 22 . Therefore, radiomics could be useful in identifying HCC patients who can benefit from immunotherapies. Acquired from routine clinical images, radiomics is a noninvasive, cost-effective method to predict the patient response, and its dynamic changes could be monitored during therapy 23 . Furthermore, the entire three-dimensional tumor landscape is captured by radiomics instead of a small portion of the tumor with spatial heterogeneity as occurs in the biopsy method 24 . Moreover, there is increasing research attention on the importance of the peritumor microenvironment 25 . Results have shown that the fusion rad-score, which consists of features from the peritumoral area, exhibited better performance than the tumor rad-score 26,27 . This indicates that the combination of peritumoral features provides more information on the tumor microenvironment, which can better reflect the biological behavior of the tumor. Therefore, in our study, we examined the entire tumor and its surrounding area, allowing us to collect more information on the tumor and its microenvironment 27,28 .
Machine learning has achieved tremendous success in recent decades. Machine learning is an effective method in the high-throughput era due to the vast amount of data that cannot be directly calculated by our human brains. In 2018, radiomics was demonstrated to be associated with clinical outcomes of cancer patients treated with anti-PD-1 and anti-PD-L1 monotherapy in a retrospective multicohort study 29 by Sun et al. Recently, Colen et al. found that a radiomics-based signature could predict the response to pembrolizumab in 57 patients with rare types of late-stage cancer with 94.7% accuracy, 97.3% sensitivity, and 90% specificity 30 . Therefore, radiomicsbased machine learning may be useful for the development of models to predict TKI-PD-1 efficacy. However, a radiomics signature extracted from baseline CTs of patients with advanced HCC treated with immune checkpoint blockade-based therapy to distinguish patients at risk of progression has not previously been reported. Thus, in this study, following the LASSO selection of 10 radiomic features, ten machine learning algorithms were applied for model construction, with SVM achieving the best performance in both the training and testing cohorts. The preliminary results showed that the SVM method was preferred regarding model construction using radiomics for advanced HCC patients treated with TKI-PD-1, while its efficacy needs to be verified in more studies.
The present study had some inherent limitations. First, whether MRI could provide more information and is more suitable for predicting TKI-PD-1 treatment efficacy needs to be determined. Second, the nature of the study was retrospective with a relatively small sample size of 55 patients because combined TKI-PD-1 therapy (different combinations) has only been applied in recent years. The overfitting problem of machine learning algorithms should be noted, and multicenter studies with large populations and subgroup analyses are warranted in the future. Third, certain tumors could not be delineated due to their infiltrative growth type, and in cases of qualified tumor imaging, only the largest tumor was segmented. Fourth, the performance of the proposed SVM model for prediction was moderate and it needs to be optimized before clinical application. Other machine learning or deep learning methods may be worthwhile in future research. Last, the integration of longitudinal multiomics (genomics, pathomics) data would be clinically significant. In conclusion, our preliminary study found that radiomics could predict TKI-PD-1 outcomes in patients with advanced HCC. Due to its noninvasive and cost-effective nature, radiomics is a promising approach to pretreatment prediction and decision-making. However, additional prospective studies with a large population obtained from multiple centers are needed to translate the present study into clinical applications.

Methods
The Ethical Review Committee of Sun Yat-sen University Cancer Center approved the study, and informed consent was obtained from all patients. The study was conducted in accordance with the principles of the Declaration of Helsinki.
Study design and patient population. The entire study design is shown in Fig. 5. Patients with advanced HCC who received TKIs orally once daily combined with a PD-1 inhibitor given intravenously (TKI-PD-1) from November 2018 to November 2019 were enrolled in this study. Participants were followed from enrollment to the date of death, loss to follow-up, or April 15, 2021.
The inclusion and exclusion criteria of patients and treatment protocols were in Supplementary method 1.1 and 1.2. The flowchart of our study is illustrated in Fig. 6.
Clinical information collection and response evaluation. Data from all enrolled patients were collected, including age, sex, virus infection, PLT, ALBI, ALT, AST, ALP, alpha-fetoprotein (AFP), and survival. The ALBI of each patient was calculated according to the following formula: ALBI = (log 10 bilirubin × 0.66) − (albumin × 0.085) 31 . In addition, imaging data, including the number and size of tumors, vascular invasion, and extrahepatic metastasis, were collected based on the abdominal CT. All best imaging responses were assessed during treatment on available follow-up images. Progressive disease (PD) was classified into the PD group, while stable disease (SD), partial response (PR), or a complete response (CR) were classified into the non-PD group according to the Response Evaluation Criteria in Solid Tumors V.1.1 guidelines 32 .
Model construction. Imaging collection and radiomic feature extractions process can be found in Supplementary method 1.3 and 1.4. After the radiomic features were obtained and filtered, ten common machine learning methods were applied to construct possible models, including SVM, NB, Rpart, Ctree, RF, KNN, neuralnet, boosting, bagging, and logistics. Due to the relatively small population, K-fold cross-validation was applied. K-fold cross-validation comprises the following steps: split the data set randomly into 5 folds (11 patients in each fold); construct the model on 4 folds, test the model on the remaining onefold, calculate the error (E) on the observations in the 1 remaining fold; and calculate the average error (AE). The equation is as follows, where k represents the number of folds and Ei indicates the error on the i th iteration: